Improvement: IBFlex stricter security matching#5303
Improvement: IBFlex stricter security matching#5303georgemac-labs wants to merge 3 commits intoportfolio-performance:masterfrom
Conversation
Extend IBFlex importer to handle cases where the reported unit is not the same as the unit the exchange quotes in (e.g. GBP vs GBX)
When dividend currency differs from security currency and no direct conversion rate exists, use fxRateToBase from the CashTransaction element to calculate cross-rates via the account's base currency. Also handles minor unit securities (e.g., USD→GBX via EUR→GBP→GBX).
georgemac-labs
force-pushed
the
ibflex-improved-security-matching
branch
from
c3cd843 to
b2d0efb
Compare
- If existing security has an ISIN, it must match - Currency must match the existing security (trades only) - Currency matching supports major/minor units
georgemac-labs
force-pushed
the
ibflex-improved-security-matching
branch
from
b2d0efb to
6d6294c
Compare
buchen
left a comment
buchen
left a comment
There was a problem hiding this comment.
let's wait for the other pull request to conclude - this one is independent but sits on top of the other pull request.
The checks wether CONID or ISIN conflict if the ticker is present make sense to me.
| * @param strictCurrencyMatch If true (for trades), only match on ISIN if currency also matches. | ||
| * If false (for dividends), allow ISIN match regardless of currency. |
There was a problem hiding this comment.
What is the rationale between separating the instrument matching whether it is a trade (purchase, sale) or a dividend transactions?
Wouldn't we get mismatches? Say a purchase creates a new security (strict match) but the corresponding dividend than is matched to the previous instrument (because it is not strictly matching).
There was a problem hiding this comment.
Trades are treated differently because the currency should always match. If it doesn't, you can be sure that should be a different security in PP.
With dividends, you don't know, because they are sometimes not paid in the listing currency. Currency mismatch is not evidence for creating a security.
And yes, you can have a mismatch. Real-world example: you have EQQQ.F (EUR) and EQQQ.L (GBP) and both pay a dividend in USD. You don't have the CONIDs stored in the WKD field. Both have the same ISIN. Currency doesn't match, but that proves nothing. Where do you assign the dividend? Currently impossible to say.
You could try to solve this using the CashTransaction's listingExchange attribute, but I think that would be more complexity, a pain to maintain (mapping table with exchange codes) and still imperfect – and all that for an edge-case.
I ignored this case, because it's not easy to get into this situation. You need to:
- Own the same instrument in multiple currencies AND
- Receive a third-currency dividend AND
- Not have the CONIDs saved
... although the joke is – that's exactly what happened to me.
Really, if you're doing non-trivial stuff with IBKR, you want CONIDs saved for all securities. And I think the IBFlex importer is getting more powerful and accurate, so in the future that should be the norm. I only ended up with securities without CONIDs due to importer issues.
There was a problem hiding this comment.
Fair. I am wondering if the user will understand the subtle difference between matching trades and dividends. I would think it is easier if it is one algorithm - and if it mixes it up, fix the meta data once and for all.
And if a new instrument is created, the user could still choose to apply them to an existing instrument:
|
|
||
| if (!isin.isEmpty() && isin.equals(security.getIsin())) | ||
| if (currency.equals(security.getCurrencyCode())) | ||
| if (isCurrencyCompatible(currency, security.getCurrencyCode())) |
There was a problem hiding this comment.
I get that point (and I understand this has nothing to do with the strict mode). I am just wondering if we take the use of the FixedEchangeRateProvider a little to far. The user could still have a DEM instrument, but I am not sure if he wants to use that instead of the creating a new EUR instrument. Maybe GBX and GBP is a special case because apparently it used interchangeably.
There was a problem hiding this comment.
I will need to look at this, but I'd say the data modelling is fundamentally a bit off. GBP and GBX are not two currencies – they are two units of the same, like Euros and Euro cents. Their relationship is quite different from DEM and EUR. But I didn't even consider opening that can of worms!
2 participants
Note: depends on #5292
When importing from IBKR, I have had some issues with transactions being matched to the wrong securities.
The importer currently starts with CONID, which is good, as it’s closest to universal and unambiguous. However, if that fails, it falls back to ISIN, and then symbol, which are ambiguous identifiers. The following scenarios are considered a match:
This PR tightens the rules as follows:
Note: I amended the existing unit test, because it contained a trade and a dividend payment where the symbols matched but the CONIDs did not. I traced the origin of the test data and it seems the dividend payment is copied from a different security: #812 (comment), so it doesn't tell us anything about real IBKR data. I only have my personal data to go on, but it's almost 9000 records, and I have zero cases where a CashTransaction has a different CONID from the associated Trade. Thus: I'm working with the assumption that they should always match.
These rules make sense to me, but I may have overlooked some facts and – as always – I’m happy to hear feedback.
Closes: #4700
Footnotes
Improvement: IBFlex handle currency unit mismatches #5291
Improvement: IBFlex support cross-currency dividend import #5292 ↩